Memory distribution in complex fitness 

landscapes 



Juan G. Diaz Ochoa 

Fachbereich 1, University of Bremen, Otto Hahn Allee, D-28359 Bremen, 

Germany 



Abstract 

In a co-evolutionary context, the survive probability of individual elements of a sys- 
tem depends on their relation with their neighbors. The natural selection process 
depends on the whole population, which is determined by local events between in- 
dividuals. Particular characteristics assigned to each individual, as larger memory, 
usually improve the individual fitness, but an agent possess also endogenous charac- 
teristics that induce to re-evaluate her fitness landscape and choose the best-suited 
kind of interaction, inducing a non absolute value of the outcomes of the interaction. 
In this work, a novel model with agents combining memory and rational choice is 
introduced, where individual choices in a complex fitness landscape induce changes 
in the distribution of the number of agents as a function of the time. In particular, 
the tail of this distribution is fat compared with distributions for agents interacting 
only with memory. 
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1 Introduction 



The emergence of cooperation without an external enforcing agency, as an ex- 
ternal coordinator, makes game theory based models attractive pQ [2] [3J. Usu- 
ally, this assumption is done on the basis that the interaction between elements 
can be absolutely denned, i.e., the fitness scale has an absolute value [3j. The 
prisoner's dilemma appears as a paradigm to model populations of interacting 
agents that are able to form clusters of cooperative individuals. What happen 
if the interaction between the elements is no more absolute? It is quite rea- 
sonable to assume non absolute values of fitness scales in interactions between 
agents. Different mechanisms can be responsible for this particular characteris- 
tic. Either the agents can suffer or induce changes in their interactions (e.g., in 
ecology by moving from one to other ecological niche) or they can simply avoid 
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the interactions with other competitors. In both cases the induced changes are 
not trivially fortuitous. For instance, some species of non-migratory birds, that 
have an ecosystem at the river of a lake, are subjected to strong competition 
against other birds of the same species due that some places are overcrowded. 
During the reproductive phase this represent a high mortality rate. But it had 
been observed that this species of birds make habitat selection, moving from 
one habitat with high mortality rate to other place with higher fitness [I]. 
This change of fitness compensates the mortality of the species. 

A frequent problem in game theory is the constant measurement unit assigned 
to the outcomes of the game [5]. Even in bimatrix games, where two different 
elements have the option to interact with different interaction matrices, the 
measurement is absolute [6] [5]. If we suppose the system is perfect isolated, 
and the elements have available only a limited interaction palette, then the 
elements have no other choice than to dispose of a single simple interaction 
matrix. In contrast, if the elements are embedded in an open system, and 
additionally have an internal reservoir that allows the search in the configu- 
ration space, the interaction matrix cannot be absolutely defined. Therefore, 
we introduce a very simple improvement in this kind of models by means of 
the definition of flexible matrices together with individual bias (preferences) 
for each agent. In this context, the novel approach presented here is the im- 
plementation of relative values of the interaction matrix. 



2 Model 

In this model, the agents can remember actions in the past and they can take 
decisions according to their stored information. Simple learning schemas can 
be defined. For instance, one schema is 'copy the best', where elements try to 
copy the best interaction from their neighborhood and implement it in their 
next interaction [7j. Memory is modeled as strips of information stored in the 
memory of agent i that are projected onto vectors of strategies in order to 
produce new actions represented by vectors a 1 [8]. 



The pair interaction is non commutative and is defined using an interaction 
matrix F. The implemented game is a prisoner's dilemma defined by a 2 x 
2 matrix, where the chain inequality T > R > P > S is obeyed ('T' for 
temptation, 'R' for reward, 'P' for punishment and 'S' for sucker). In this 
model only pair interactions are allowed. The interaction for an agent and 
her adversary can be either cooperate -equivalent in statistical mechanics to 
spin up- or defect -equivalent to spin down-. The vector notes each strategy 
a. If a 1 is the interaction of the agent i and is the interaction of agent j, 
then, the total utility matrix for the agent i is given by W 1 = a^Fa\ and the 
total utility of the agent i is defined as /' = J2j [5]. Under the imposed 
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constraints imposed by F, it is rational for the two agents to defect. But also 
it would be of mutual advantage for the player to establish cooperation in a 
long run. In the model the interactions are implemented in such a way that 
the agents have perfect information from the past, but no information of the 
instantaneous interaction of the adversary. 

The definition of non absolute values in the interactions is made by means 
of an extended interaction matrix F p . Furthermore, an additional vector (k 
defines the preferred interaction of the agent i. This vector points into each 
one of the interaction sub-matrixes to be fixed in each interaction. The utility 
matrix for the agent i has in this case the following form 



The games are characterized by three kinds of sub-matrices. One sub-matrix 
is defined under the restriction 2R > T + S. The second sub-matrix is defined 
as a chicken game, which means, the mutual defection is the worst possible 
outcome. Therefore, the values for this sub-matrix are T' > R' > S' > P' . 
The values were arbitrary defined, but in such a away that T' < R', i.e., 
'temptation' is not dramatically different to 'reward'. Additionally, T' < T, 
i.e. is a low risk game. A third sub-matrix representing a constant interaction 
is introduced, with the condition T" = R" = S" — P" — 1. This last sub- 
matrix implies for the agents a total risk aversion, but low incomes. Hence, 
the agents interacting in the present model could basically choose between 
more or less risk. Simultaneously, these three fundamental options allow the 
game to switch between stable and unstable equilibrium states. 

This model assumes a number of N agents situated in a lattice (see fig. [TJ. 
Given that this is a square lattice, each element can only interact with its 
four nearest neighbors, according to the Von Neumann Schema. The memory 
size M of each agent is the main observable of the system. The acceptance 
or rejection depends on the utility f l obtained by agent i. Naturally, the 
evolution of the memory size of the individuals automatically implies changes 
in the local strategies of the elements. At t — there is a random distribution 
of agents with low memories (M l < 2). If the total utility /* of the element i 
is smaller than the utility of the adjunct opponents j, then a mutation of the 
memory size of i, M\ occurs, i.e. M % — > (M + 1)*. This mutation takes place 
at frequencies u m , that are proportional to the memory size. The reason is, 
the assumption that the mutation probability in complex elements does not 
take place with the same frequency as the mutation in much simpler elements. 
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3 Results 



We make a qualitative analysis of the distribution of memories when the agents 
have no preferences, i.e., when the interactions are given by the canonical ma- 
trix F = (3,0,5,2). After a long simulation time a relative homogeneous 
mixture of elements with low memory suffers a transition. The mixture be- 
comes inhomogeneous and forms a structure that alternates between different 
memory components. The colors in fig. [1] symbolize different memory size. 
Blue corresponds to agents with low memories, whereas yellow symbolizes 
agents with high memory. After long iteration time non-symmetric patterns 
emerge. Although these patterns do not have a clear structure, they are also 
not completely random. We compare this result with the behavior of agents 
allowed to choose into the flexible fitness landscape (right side figure). At the 
same time scale the distribution of agents with low memory dominates over 
the agents with high memory, and only few non symmetric patterns with high 
memory agents appears. The time dependence of the distribution of mem- 




Fig. 1. Snapshot of the distribution of memories. The figure on the left side corre- 
sponds to prisoner's dilemma interactions. The right side figure represents mixed 
interactions and agents with individual choices. 

ory shows a rapid growth and subsequent extinction after some characteristic 
time. The mechanism is in such case very simple: if the fitness of the agents 
with memory M is large enough, then they do not find any resistance and can 
replace other elements in the neighborhood and spread into the system. If a 
new class of elements with a much better fitness and other memory size M' 
appears, these new elements start to spread. Eventually the agents with less 
fitness, i.e. agents with memory M, start an extinction process. The extinction 
rate strongly depends on the interaction each agent chooses. When the agents 
interact with flexible matrices < F >, which is equivalent to agents with 
choices into the extended fitness landscape F p , the fluctuation-dissipation is 
no more in equilibrium. With less dissipation the agents increase their chance 
to reproduce themselves. This fact is translated in the existence of self-similar 
processes that take place in different time regimes. Therefore, the distribu- 
tion in time of the number of agents with memory M, N[M], have a fat tail 
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when the interactions are more flexible than the simple interaction governed 
by the prisoner's dilemma. We can then concentrate us in a specific memory. 




t 



Fig. 2. Distribution in time of the total population of the system < JV[M] > as 
a function of the time for flexible interactions < F > (agents with choices into 
the extended fitness landscape F p ) and prisoner's dilemma F.. The inset shows the 
non- normalized distribution of the number of agents with memory M = 6 and fit. 

The distribution function is asymmetric and can be fitted with the following 
distribution [9] 

I A n [Ml e ~MM]/2r 

MM](r) = ^ °l 16 r5/2 , (2) 

where A [M] is a fit parameter for each memory size. Given that the fit is 
valid for a long time scale, the time parameter r is defined as r = where 
T is the characteristic frequency where a growth and subsequent extinction 
of the referred memory size takes place. In the present computations To ~ 
T, where T is the period where the computations were done. This fit is a 
reasonable approximation to the shape of the density function and is shown in 
the inset of Fig. [2j In the same figure, the average of the population of different 
memories < iV[M] > is shown. In long time regimes the suppression of the 
dissipation in the case of flexible interactions allows a relative stability in the 
total population (more diversity), whereas for the simple prisoner's dilemma 
a mass extinction takes place. 



4 Sumary 

The present work describes quantitatively the ability that an agent has to 
choose in a complex energy landscape. This characteristic of the agents cannot 
simply be ignored and must be implemented including individual memory 
features. 
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To sum up, the present results are a heuristic approximation to describe sys- 
tems consisting of agents that can modify their fitness. Two main aspects can 
be extracted from the present model: first, the possibility to obtain communi- 
ties with high diversity and inhomogeneous spatial distribution. Second, this 
diversity depends on the ability that each agent shows to choose the kind of 
interaction she needs. 

I want to thank H. Fort for very useful comments. 
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